<!DOCTYPE html>
<html lang="en">
  <head>
  <meta charset="utf-8" />
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">


  <title>kubernetes 下实现socket.io 的集群模式-blog.qikqiak.com|阳明的博客|Kubernetes|Docker|Python|Golang|Cloud Native</title>
  <meta property="og:title" content="kubernetes 下实现socket.io 的集群模式" />
  <meta name="twitter:title" content="kubernetes 下实现socket.io 的集群模式" />

  <meta name="description" content="socket.io 单节点模式是很容易部署的，但是往往在生产环境一个节点不能满足业务需求，况且还要保证节点挂掉的情况仍能正常提供服务，所以多节点模式就成为了生成环境的一种必须的部署模式。

本文将介绍如何在kubernetes 集群上部署多节点的socket.io服务。

文章中涉及到的代码可以前往https://github.com/cnych/k8s-socketio-cluster-demo查看。

">
  <meta property="og:description" content="socket.io 单节点模式是很容易部署的，但是往往在生产环境一个节点不能满足业务需求，况且还要保证节点挂掉的情况仍能正常提供服务，所以多节点模式就成为了生成环境的一种必须的部署模式。

本文将介绍如何在kubernetes 集群上部署多节点的socket.io服务。

文章中涉及到的代码可以前往https://github.com/cnych/k8s-socketio-cluster-demo查看。

">
  <meta name="twitter:description" content="socket.io 单节点模式是很容易部署的，但是往往在生产环境一个节点不能满足业务需求，况且还要保证节点挂掉的情况仍能正常提供服务，所以多节点模式就成为了生成环境的一种必须的部署模式。

本文将介绍如何在kubernetes 集群上部署多节点的socket.io服务。

文章中涉及到的代码可以前往https://github.com/cnych/k8s-socketio-cluster-demo …">
  <meta name="author" content=""/>
  <link href='https://blog.qikqiak.com/img/favicon.ico' rel='icon' type='image/x-icon'/>
  <meta property="og:image" content="https://blog.qikqiak.com/img/avatar.jpeg" />
  <meta name="twitter:image" content="https://blog.qikqiak.com/img/avatar.jpeg" />
  <meta name="twitter:card" content="summary" />
  <meta property="og:url" content="https://blog.qikqiak.com/post/socketio-multiple-nodes-in-kubernetes/" />
  <meta property="og:type" content="website" />
  <meta property="og:site_name" content="阳明的博客" />

  <meta name="generator" content="Hugo 0.34" />
  <link rel="canonical" href="https://blog.qikqiak.com/post/socketio-multiple-nodes-in-kubernetes/" />
  <link rel="alternate" href="https://blog.qikqiak.com/index.xml" type="application/rss+xml" title="阳明的博客">
  <link href="https://cdn.bootcss.com/KaTeX/0.7.1/katex.min.css" rel="stylesheet">
  <link href="https://cdn.bootcss.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet">
  <link href="https://cdn.bootcss.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet">
  <link rel="stylesheet" href="https://blog.qikqiak.com/css/main.css?t=1235" />
  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Lora:400,700,400italic,700italic" />
  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" />
  <link rel="stylesheet" href="https://blog.qikqiak.com/css/pygment_highlights.css" />
  <link rel="stylesheet" href="https://blog.qikqiak.com/css/highlight.min.css" />
  <link rel="stylesheet" href="https://blog.qikqiak.com/css/prism.css?t=123" />
  <link rel="stylesheet" href="https://blog.qikqiak.com/css/search.css" />
<meta name="keywords" content="kubernetes, socket.io, cluster, 集群, nodejs">
<script>
var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "https://hm.baidu.com/hm.js?970de9920f6089c287a7fd366138d4e9";
  var s = document.getElementsByTagName("script")[0];
  s.parentNode.insertBefore(hm, s);
})();
</script>



<link href="https://cdn.bootcss.com/photoswipe/4.1.1/photoswipe.min.css" rel="stylesheet">
<link href="https://cdn.bootcss.com/photoswipe/4.1.1/default-skin/default-skin.min.css" rel="stylesheet">



<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

<div class="pswp__bg"></div>

<div class="pswp__scroll-wrap">
    
    <div class="pswp__container">
      <div class="pswp__item"></div>
      <div class="pswp__item"></div>
      <div class="pswp__item"></div>
    </div>
    
    <div class="pswp__ui pswp__ui--hidden">
    <div class="pswp__top-bar">
      
      <div class="pswp__counter"></div>
      <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>
      <button class="pswp__button pswp__button--share" title="Share"></button>
      <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>
      <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>
      
      
      <div class="pswp__preloader">
        <div class="pswp__preloader__icn">
          <div class="pswp__preloader__cut">
            <div class="pswp__preloader__donut"></div>
          </div>
        </div>
      </div>
    </div>
    <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
      <div class="pswp__share-tooltip"></div>
    </div>
    <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
    </button>
    <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
    </button>
    <div class="pswp__caption">
      <div class="pswp__caption__center"></div>
    </div>
    </div>
    </div>
</div>

<script>
window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date;
ga('create', 'UA-69668147-3', 'auto');
ga('send', 'pageview');
</script>
<script async src='//www.google-analytics.com/analytics.js'></script>

</head>

  <body>
    <nav class="navbar navbar-default navbar-fixed-top navbar-custom">
  <div class="container-fluid">
    <div class="navbar-header">
      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#main-navbar">
        <span class="sr-only">Toggle navigation</span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
        <span class="icon-bar"></span>
      </button>
      <a class="navbar-brand" href="https://blog.qikqiak.com/">阳明的博客</a>
    </div>

    <div class="collapse navbar-collapse" id="main-navbar">
      <ul class="nav navbar-nav navbar-right">
        
          
            <li>
              <a title="Home" href="https://ydzs.io">Home</a>
            </li>
          
        
          
            <li class="navlinks-container">
              <a class="navlinks-parent" href="javascript:void(0)">Categories</a>
              <div class="navlinks-children">
                
                  <a href="https://blog.qikqiak.com/page/archive/">Archive</a>
                
                  <a href="https://blog.qikqiak.com/tags">tags</a>
                
                  <a href="https://blog.qikqiak.com/tags/kubernetes">kubernetes</a>
                
                  <a href="https://blog.qikqiak.com/tags/python">python</a>
                
                  <a href="https://blog.qikqiak.com/tags/django">django</a>
                
                  <a href="https://blog.qikqiak.com/tags/ops">devops</a>
                
              </div>
            </li>
          
        
          
            <li class="navlinks-container">
              <a class="navlinks-parent" href="javascript:void(0)">海马学院</a>
              <div class="navlinks-children">
                
                  <a href="https://www.haimaxy.com/course/6y02wp/?utm_source=blog&amp;utm_campaign=referral&amp;utm_medium=topmenu">Python微服务实战</a>
                
                  <a href="https://www.haimaxy.com/course/pww13p/?utm_source=blog&amp;utm_campaign=referral&amp;utm_medium=topmenu">Webpack3.x 入门</a>
                
                  <a href="https://www.haimaxy.com/course/pjrqxm/?utm_source=blog&amp;utm_campaign=referral&amp;utm_medium=topmenu">Kubernetes 集群搭建</a>
                
              </div>
            </li>
          
        
          
            <li>
              <a title="About" href="https://blog.qikqiak.com/page/about/">About</a>
            </li>
          
        

        

        

        
          <li>
            <a href="#modalSearch" data-toggle="modal" data-target="#modalSearch" style="outline: none;">
              <span class="hidden-sm hidden-md hidden-lg">Search</span> <span id="searchGlyph" class="glyphicon glyphicon-search"></span>
            </a>
          </li>
          

      </ul>
    </div>

    <div class="avatar-container">
      <div class="avatar-img-border">
        
          <a title="阳明的博客" href="https://blog.qikqiak.com/">
            <img class="avatar-img" src="https://blog.qikqiak.com/img/avatar.jpeg" alt="阳明的博客" />
          </a>
        
      </div>
    </div>

  </div>
</nav>





  <div id="modalSearch" class="modal fade" role="dialog">
    <div class="modal-dialog">
      <div class="modal-content">
        <div class="modal-header">
          <button type="button" class="close" data-dismiss="modal">&times;</button>
          <h4 class="modal-title">Search blog.qikqiak.com</h4>
        </div>
        <div class="modal-body">
            
<div class="aa-input-container" id="aa-input-container">
    <input type="search" id="aa-search-input" class="aa-input-search" placeholder="Search for titles or URIs..." name="search" autocomplete="off" />
    <svg class="aa-input-icon" viewBox="654 -372 1664 1664">
        <path d="M1806,332c0-123.3-43.8-228.8-131.5-316.5C1586.8-72.2,1481.3-116,1358-116s-228.8,43.8-316.5,131.5  C953.8,103.2,910,208.7,910,332s43.8,228.8,131.5,316.5C1129.2,736.2,1234.7,780,1358,780s228.8-43.8,316.5-131.5  C1762.2,560.8,1806,455.3,1806,332z M2318,1164c0,34.7-12.7,64.7-38,90s-55.3,38-90,38c-36,0-66-12.7-90-38l-343-342  c-119.3,82.7-252.3,124-399,124c-95.3,0-186.5-18.5-273.5-55.5s-162-87-225-150s-113-138-150-225S654,427.3,654,332  s18.5-186.5,55.5-273.5s87-162,150-225s138-113,225-150S1262.7-372,1358-372s186.5,18.5,273.5,55.5s162,87,225,150s113,138,150,225  S2062,236.7,2062,332c0,146.7-41.3,279.7-124,399l343,343C2305.7,1098.7,2318,1128.7,2318,1164z" />
    </svg>
</div>
<script src="https://res.cloudinary.com/jimmysong/raw/upload/rootsongjc-hugo/algoliasearch.min.js"></script>
<script src="https://res.cloudinary.com/jimmysong/raw/upload/rootsongjc-hugo/autocomplete.min.js"></script>

<script>
var client = algoliasearch("1JDRAS0AZR", "8804ac109158bb3bb60d74ce98fa332f");
var index = client.initIndex('prod_blog');

autocomplete('#aa-search-input',
{ hint: false}, {
    source: autocomplete.sources.hits(index, {hitsPerPage: 5}),
    
    displayKey: 'name',
    
    templates: {
        
        suggestion: function(suggestion) {
            return '<span>' + '<a href="https://blog.qikqiak.com/post/' + suggestion.slug + '">' +
            suggestion._highlightResult.title.value + '</a></span>';
        }
    }
});
</script>
        </div>
        <div class="modal-footer">
          <button type="button" class="btn btn-default" data-dismiss="modal">close</button>
        </div>
      </div>
    </div>
  </div>

    
  
  
  




  
    <div id="header-big-imgs" data-num-img=1 data-img-src-1="https://blog.qikqiak.com/img/posts/23498828_2151213871773296_819155850823204864_n.jpg" data-img-desc-1="A pal and a skateboard and headed outside in South Australia."></div>
  

  <header class="header-section has-img">
    
      <div class="intro-header big-img">
        
        <div class="container">
          <div class="row">
            <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
              <div class="post-heading">
                <h1>kubernetes 下实现socket.io 的集群模式</h1>
                  
                  
                    <span class="post-meta">
  Posted on November 21, 2017
  
</span>


                  
              </div>
            </div>
          </div>
        </div>
        <span class="img-desc" style="display: inline;"></span>
      </div>
    
    <div class="intro-header no-img">
      
      <div class="container">
        <div class="row">
          <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
            <div class="post-heading">
              <h1>kubernetes 下实现socket.io 的集群模式</h1>
                
                
                  <span class="post-meta">
  Posted on November 21, 2017
  
</span>


                
            </div>
          </div>
        </div>
      </div>
    </div>
  </header>


    
<div class="container" role="main">
  <div class="row">
    <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">

      <div>
          
          
          <h5 id="tags" style="margin-top: 0px;">标签:
            
                <a href="https://blog.qikqiak.com/tags/kubernetes/">kubernetes</a> &nbsp;
            
                <a href="https://blog.qikqiak.com/tags/socket.io/">socket.io</a> &nbsp;
            
                <a href="https://blog.qikqiak.com/tags/cluster/">cluster</a> &nbsp;
            
          </h5>
          
      </div>

      <article role="main" class="blog-post">
        <p><code>socket.io</code> 单节点模式是很容易部署的，但是往往在生产环境一个节点不能满足业务需求，况且还要保证节点挂掉的情况仍能正常提供服务，所以多节点模式就成为了生成环境的一种必须的部署模式。</p>

<p>本文将介绍如何在kubernetes 集群上部署多节点的<code>socket.io</code>服务。</p>

<p>文章中涉及到的代码可以前往<a href="https://github.com/cnych/k8s-socketio-cluster-demo">https://github.com/cnych/k8s-socketio-cluster-demo</a>查看。</p>

<p></p>

<h3 id="问题">问题</h3>

<p>现在正在准备将线上环境一步步迁移到<a href="https://blog.qikqiak.com/tags/kubernetes/">kubernetes 集群</a>上，这样我们可以根据实际情况部署多个POD 来提供服务，但是<code>socket.io</code>服务并不是单纯的无状态应用，只需要将POD 部署成多个就可以正常提供服务了，因为其底层需要建立很多连接来保持长连接，但是这样的话上一个请求可能会被路由到一个POD，下一个请求则很有可能会被路由到另外一个POD 中去了，这样就会出现错误了，如下图：</p>

<p><img src="https://blog.qikqiak.com/img/posts/WX20171121-135349.png" alt="socket-io errors" /></p>

<p>从上面的错误中我们可以看出是有的请求找不到对应的Session ID，也证明了上面提到的引起错误的原因。</p>

<h3 id="解决方法">解决方法</h3>

<p>我们从<a href="https://socket.io/docs/using-multiple-nodes/">socket.io 官方文档</a>中可以看到对于多节点的介绍，其中通过<code>Nginx</code>的<strong>ip_hash</strong> 配置用得比较多，同一个ip 访问的请求通过hash 计算过后会被路由到相同的后端程序去，这样就不会出现上面的问题了。我们这里是部署在<code>kubernetes</code>集群上面的，通过<code>traefik ingress</code>来连接外部和集群内部间的请求的，所以这里中间就省略了<code>Nginx</code>这一层，当然你也可以多加上这一层，但是这样显然从架构上就冗余了，而且还有更好的解决方案的：<strong>sessionAffinity</strong>（也称会话亲和力）</p>

<blockquote>
<p>什么是sessionAffinity？
sessionAffinity是一个功能，将来自同一个客户端的请求总是被路由回服务器集群中的同一台服务器的能力。</p>
</blockquote>

<p>在<code>kubernetes</code>中启用<code>sessionAffinity</code>很简单，只需要简单的<code>Service</code>中配置即可：</p>

<pre><code class="language-json">service.spec.sessionAffinity = &quot;ClientIP&quot;
</code></pre>

<p>默认情况下sessionAffinity=None，会随机选择一个后端进行路由转发的，设置成<code>ClientIP</code>后就和上面的<code>ip_hash</code>功能一样了，由于我们使用的是<code>traefik ingress</code>，这里还需要在<code>Service</code>中添加一个<code>traefik</code>的<code>annotation</code>：</p>

<pre><code class="language-yaml">apiVersion: v1
kind: Service
metadata:
  name: socket-demo
  namespace: kube-apps
  annotations:
    traefik.backend.loadbalancer.stickiness: &quot;true&quot;
    traefik.backend.loadbalancer.stickiness.cookieName: &quot;socket&quot;
  labels:
    k8s-app: socket-demo
spec:
  sessionAffinity: &quot;ClientIP&quot;
  ports:
    - name: socketio
      port: 80
      protocol: TCP
      targetPort: 3000
  selector:
    k8s-app: socket-demo
</code></pre>

<p>注意上面的<strong>annotations</strong>和<strong>sessionAffinity</strong>两项配置，然后我们再来看看我们的<code>socket.io</code>服务吧</p>

<p><img src="https://blog.qikqiak.com/img/posts/WX20171121-140324.png" alt="socket.io" />
已经正常了吧，注意看上面打印出来的<code>hostname</code>都是一样的，因为我们这里去访问的都是来自同一个IP，多刷新几次是不是还是这样，证明上面的<code>sessionAffinity</code>配置生效了。</p>

<p>如果是另外的地方去访问，会路由到不一样的后端去吗？我们这里启用一个代理来测试下：
<img src="https://blog.qikqiak.com/img/posts/WX20171121-10000.png" alt="socket.io" />
从上图中打印出来的<code>hostname</code>可以看出两个请求被路由到了不同的POD 中，但是现在又有一个新的问题了：<strong>绘制的图形并没有被广播出去</strong>，这是为什么呢？其实在上面提到的<code>socket.io 官方文档中已经提到过了</code>：</p>

<blockquote>
<p>Now that you have multiple Socket.IO nodes accepting connections, if you want to broadcast events to everyone (or even everyone in a certain room) you’ll need some way of passing messages between processes or computers.</p>

<p>The interface in charge of routing messages is what we call the Adapter. You can implement your own on top of the socket.io-adapter (by inheriting from it) or you can use the one we provide on top of Redis: socket.io-redis:</p>
</blockquote>

<pre><code class="language-javascript">var io = require('socket.io')(3000);
var redis = require('socket.io-redis');
io.adapter(redis({ host: 'localhost', port: 6379 }));
</code></pre>

<p>总结起来就是你如果想在进程间或者节点之前发送信息，那么就需要实现自己实现一个<code>socket.io-adapter</code>或者利用官方提供的<code>socket.io-redis</code>。</p>

<p>我们这里利用<code>socket.io-redis</code> 这个adapter 来实现消息的广播，最终的服务端代码如下：</p>

<pre><code class="language-javascript">const express = require('express');
const socketRedis = require('socket.io-redis');
const os = require('os');
const app = express();
const http = require('http').Server(app);
const io = require('socket.io')(http);
const port = process.env.PORT || 3000;

app.use(express.static(__dirname + '/static'));

app.engine('html', require('ejs').renderFile);
app.set('view engine', 'html');

app.get('/', function (req, res) {
    res.render('index', {
        'os': os.hostname()
    });
});

function onConnection(socket){
  socket.on('drawing', (data) =&gt; socket.broadcast.emit('drawing', data));
}

io.adapter(socketRedis({host: 'redis', port: 6379}));
io.on('connection', onConnection);

http.listen(port, () =&gt; console.log('listening on port ' + port));
</code></pre>

<p>部署在<code>kubernetes</code>集群上的<code>yaml</code>文件如下：</p>

<pre><code class="language-yaml">apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: socket-demo
  namespace: kube-apps
  labels:
    k8s-app: socket-demo
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: socket-demo
    spec:
      containers:
        - image: cnych/socketdemo:k8s
          imagePullPolicy: Always
          name: socketdemo
          ports:
            - containerPort: 3000
              protocol: TCP
          resources:
            limits:
              cpu: 100m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 50Mi

---
apiVersion: v1
kind: Service
metadata:
  name: socket-demo
  namespace: kube-apps
  annotations:
    traefik.backend.loadbalancer.stickiness: &quot;true&quot;
    traefik.backend.loadbalancer.stickiness.cookieName: &quot;socket&quot;
  labels:
    k8s-app: socket-demo
spec:
  sessionAffinity: None
  ports:
    - name: socketio
      port: 80
      protocol: TCP
      targetPort: 3000
  selector:
    k8s-app: socket-demo
</code></pre>

<p>现在看看最终的效果吧：
<img src="https://blog.qikqiak.com/img/posts/WX20171121-141623.png" alt="socket.io cluster" /></p>

<p>不同节点间也可以传递数据了，到这里我们就实现了在<code>kubernetes</code>集群下部署<code>socket.io</code>多节点。</p>

<blockquote>
<p>上面的根据<code>traefik.backend.loadbalancer.stickiness.cookieName</code>来进行路由的规则在测试环境生效了，在线上没生效，可能这个地方有什么问题？</p>

<p>上面没有生效是因为客户端连接<code>socket.io</code>的协议的时候没有使用<strong>polling</strong>造成的，客户端连接<code>socket.io</code>要按照标准的方式指定trasports=[&lsquo;polling&rsquo;, &lsquo;websocket&rsquo;]</p>

<p><code>sessionAffinity</code> 与 <code>traefik</code>设置cookieName的方式貌似不能同时存在，如果遇到不生效的，将<code>sessionAffinity</code>设置为<strong>None</strong> ，只保留<code>traefik</code>的annotaions。</p>

<p>在使用<code>socket.io-redis</code>的时候一定要注意，在<code>join</code>和<code>leave</code>房间的时候一定要使用<code>adapter</code>提供的<code>remoteJoin</code>和<code>remoteLeave</code>方法，不然多个节点间的数据同步有问题，这个被坑了好久&hellip;&hellip;</p>
</blockquote>

<h3 id="参考文档">参考文档</h3>

<ul>
<li><a href="https://docs.traefik.io/basics/#sticky-sessions">traefik sticky-sessions</a></li>
<li><a href="https://kubernetes.io/docs/concepts/services-networking/service/">kubernetes service</a></li>
<li><a href="https://socket.io/docs/using-multiple-nodes/">socket.io docs</a></li>
<li><a href="https://github.com/socketio/socket.io-redis">socket.io redis</a></li>
</ul>

<p>扫描下面的二维码(或微信搜索<code>k8s技术圈</code>)关注我们的微信公众帐号，在微信公众帐号中回复 <strong>加群</strong> 即可加入到我们的 kubernetes 讨论群里面共同学习。
<img src="https://blog.qikqiak.com/img/posts/qrcode_for_gh_d6dd87b6ceb4_430.jpg" alt="qrcode" /></p>
      </article>

      
        

<h3>相关文章</h3>
<ul style="margin-bottom: 25px;">
    
    <li><a href="https://blog.qikqiak.com/post/manual-install-high-available-kubernetes-cluster/">手动搭建高可用的kubernetes 集群</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/install-docker-registry-harbor-in-kubernetes/">在kubernetes 集群上搭建docker 私有仓库Harbor</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/update-kubernetes-dashboard-more-secure/">kubernetes dashboard 升级之路</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/kubernetes-rollout-update/">Kubernetes Deployment滚动升级</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/kubernetes-monitor-prometheus-grafana/">Kubernetes使用Prometheus搭建监控平台</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/first-use-helm-on-kubernetes/">Kubernetes Helm 初体验</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/add-authorization-for-kubernetes-dashboard/">为kubernetes dashboard访问用户添加权限控制</a></li>
    
    <li><a href="https://blog.qikqiak.com/post/understand-kubernetes-configmap-and-secrets/">kubernetes ConfigMap 和 Secrets</a></li>
    
</ul>

      

      
      <ul class="pager blog-pager">
        
          <li class="previous">
            <a href="https://blog.qikqiak.com/post/install-docker-registry-harbor-in-kubernetes/" data-toggle="tooltip" data-placement="top" title="在kubernetes 集群上搭建docker 私有仓库Harbor">&larr; Previous Post</a>
          </li>
        
        
          <li class="next">
            <a href="https://blog.qikqiak.com/post/kubernetes-logs-collect/" data-toggle="tooltip" data-placement="top" title="kubernetes 日志收集方案">Next Post &rarr;</a>
          </li>
        
      </ul>
      

      

      
      <div id="git-comments"></div>
      <link rel="stylesheet" href="https://imsun.github.io/gitment/style/default.css">
      <script src="https://imsun.github.io/gitment/dist/gitment.browser.js"></script>
      <script>
      var gitment = new Gitment({
        id: 'socketio-multiple-nodes-in-kubernetes',
        owner: 'cnych',
        repo: 'blog',
        oauth: {
          client_id: 'bdb76dbb2e9d0786e350',
          client_secret: 'b454b2a08013fd0e32013be7a63fa8fcb262b6c4',
        }
      })
      gitment.render('git-comments')
      </script>
      

    </div>
  </div>
</div>

    <footer>
  <div class="container">
    <div class="row">
      <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
        <ul class="list-inline text-center footer-links">
          
              <li>
                <a href="mailto:icnych@gmail.com" title="Email me">
                  <span class="fa-stack fa-lg">
                    <i class="fa fa-circle fa-stack-2x"></i>
                    <i class="fa fa-envelope fa-stack-1x fa-inverse"></i>
                  </span>
                </a>
              </li>
          
              <li>
                <a href="https://github.com/cnych" title="GitHub">
                  <span class="fa-stack fa-lg">
                    <i class="fa fa-circle fa-stack-2x"></i>
                    <i class="fa fa-github fa-stack-1x fa-inverse"></i>
                  </span>
                </a>
              </li>
          
              <li>
                <a href="https://weibo.com/cnych" title="微博">
                  <span class="fa-stack fa-lg">
                    <i class="fa fa-circle fa-stack-2x"></i>
                    <i class="fa fa-weibo fa-stack-1x fa-inverse"></i>
                  </span>
                </a>
              </li>
          
              <li>
                <a href="https://instagram.com/cnych" title="Instagram">
                  <span class="fa-stack fa-lg">
                    <i class="fa fa-circle fa-stack-2x"></i>
                    <i class="fa fa-instagram fa-stack-1x fa-inverse"></i>
                  </span>
                </a>
              </li>
          
          
          <li>
            <a href="https://blog.qikqiak.com/index.xml" title="RSS">
              <span class="fa-stack fa-lg">
                <i class="fa fa-circle fa-stack-2x"></i>
                <i class="fa fa-rss fa-stack-1x fa-inverse"></i>
              </span>
            </a>
          </li>
          
        </ul>
        <p class="credits copyright text-muted">
          

          &nbsp;&bull;&nbsp;
          2018

          
            &nbsp;&bull;&nbsp;
            <a href="https://blog.qikqiak.com/">阳明的博客</a>
          
        </p>
        
        <p class="credits theme-by text-muted">
          <a href="http://gohugo.io">Hugo v0.34</a> powered &nbsp;&bull;&nbsp; Theme by <a href="http://deanattali.com/beautiful-jekyll/">Beautiful Jekyll</a> adapted to <a href="https://github.com/halogenica/beautifulhugo">Beautiful Hugo</a>
          
        </p>
      </div>
    </div>
  </div>
</footer>

<script src="https://cdn.bootcss.com/KaTeX/0.7.1/katex.min.js"></script>
<script src="https://cdn.bootcss.com/KaTeX/0.7.1/contrib/auto-render.min.js"></script>
<script src="https://cdn.bootcss.com/jquery/1.12.4/jquery.min.js"></script>
<script src="https://cdn.bootcss.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
<script src="https://blog.qikqiak.com/js/main.min.js"></script>
<script src="https://blog.qikqiak.com/js/highlight.min.js"></script>
<script> hljs.initHighlightingOnLoad(); </script>
<script> renderMathInElement(document.body); </script>
<script src="https://blog.qikqiak.com/js/prism.js?t=123"></script>
<script src="https://cdn.bootcss.com/photoswipe/4.1.1/photoswipe.min.js"></script>
<script src="https://cdn.bootcss.com/photoswipe/4.1.1/photoswipe-ui-default.min.js"></script>
<script src="https://blog.qikqiak.com/js/load-photoswipe.min.js"></script>



<script async src="https://www.googletagmanager.com/gtag/js?id=UA-69668147-3"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());
  gtag('config', 'UA-69668147-3');
</script>

  </body>
</html>

